On the optimal analysis of the collision probability tester

The collision probability tester, introduced by Goldreich and Ron (ECCC, TR00-020, 2000), distinguishes the uniform distribution over [n] from any distribution that is ǫ-far from this distribution using poly(1/ǫ) ·√n samples. While the original analysis established only an upper bound of O(1/ǫ) · √n on the sample complexity, a recent analysis of Diakonikolas, Gouleakis, Peebles, and Price (ECCC, TR16-178, 2016) established the optimal upper bound of O(1/ǫ) · √ n. In this note we survey their analysis, while highlighting the sources of improvement. Specifically: 1. While the original analysis reduces the testing problem to approximating the collision probability of the unknown distribution up to a 1 + ǫ factor, the improved analysis capitalizes on the fact that the latter problem needs only be solved “at the extreme” (i.e., it suffices to distinguish the uniform distribution, which has collision probability 1/n, from any distribution that has collision probability exceeding (1 + 4ǫ)/n). 2. While the original analysis provides an almost optimal analysis of the variance of the estimator when ǫ = Ω(1), a more careful analysis yields a significantly better bound for the case of ǫ = o(1), which is the case that is relevant here.